---

title: TabModel


keywords: fastai
sidebar: home_sidebar

summary: "This is an implementation created by Ignacio Oguiza based on fastai's TabularModel - oguiza@gmail.com. I build it so that it's easy to change the head of the model, something that is particularly interesting when building hybrid models."
description: "This is an implementation created by Ignacio Oguiza based on fastai's TabularModel - oguiza@gmail.com. I build it so that it's easy to change the head of the model, something that is particularly interesting when building hybrid models."
nb_path: "nbs/120_models.TabModel.ipynb"
---
<!--

#################################################
### THIS FILE WAS AUTOGENERATED! DO NOT EDIT! ###
#################################################
# file to edit: nbs/120_models.TabModel.ipynb
# command to build the docs after a change: nbdev_build_docs

-->

<div class="container" id="notebook-container">
        
    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="kn">from</span> <span class="nn">tsai.data.tabular</span> <span class="kn">import</span> <span class="o">*</span>
<span class="kn">from</span> <span class="nn">tsai.models.utils</span> <span class="kn">import</span> <span class="o">*</span>
</pre></div>

    </div>
</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

<div class="output_wrapper">
<div class="output">

<div class="output_area">


<div class="output_markdown rendered_html output_subarea ">
<h2 id="TabModel" class="doc_header"><code>class</code> <code>TabModel</code><a href="https://github.com/timeseriesAI/tsai/tree/main/tsai/models/TabModel.py#L10" class="source_link" style="float:right">[source]</a></h2><blockquote><p><code>TabModel</code>(<strong><code>emb_szs</code></strong>, <strong><code>n_cont</code></strong>, <strong><code>c_out</code></strong>, <strong><code>layers</code></strong>=<em><code>None</code></em>, <strong><code>fc_dropout</code></strong>=<em><code>None</code></em>, <strong><code>embed_p</code></strong>=<em><code>0.0</code></em>, <strong><code>y_range</code></strong>=<em><code>None</code></em>, <strong><code>use_bn</code></strong>=<em><code>True</code></em>, <strong><code>bn_final</code></strong>=<em><code>False</code></em>, <strong><code>bn_cont</code></strong>=<em><code>True</code></em>, <strong><code>lin_first</code></strong>=<em><code>False</code></em>, <strong><code>act</code></strong>=<em><code>ReLU(inplace=True)</code></em>, <strong><code>skip</code></strong>=<em><code>False</code></em>) :: <a href="/models.TabFusionTransformer.html#Sequential"><code>Sequential</code></a></p>
</blockquote>
<p>Basic model for tabular data.</p>

</div>

</div>

</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

<div class="output_wrapper">
<div class="output">

<div class="output_area">


<div class="output_markdown rendered_html output_subarea ">
<h2 id="TabBackbone" class="doc_header"><code>class</code> <code>TabBackbone</code><a href="https://github.com/timeseriesAI/tsai/tree/main/tsai/models/TabModel.py#L25" class="source_link" style="float:right">[source]</a></h2><blockquote><p><code>TabBackbone</code>(<strong><code>emb_szs</code></strong>, <strong><code>n_cont</code></strong>, <strong><code>embed_p</code></strong>=<em><code>0.0</code></em>, <strong><code>bn_cont</code></strong>=<em><code>True</code></em>) :: <code>Module</code></p>
</blockquote>
<p>Same as <code>nn.Module</code>, but no need for subclasses to call <code>super().__init__</code></p>

</div>

</div>

</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

<div class="output_wrapper">
<div class="output">

<div class="output_area">


<div class="output_markdown rendered_html output_subarea ">
<h2 id="TabHead" class="doc_header"><code>class</code> <code>TabHead</code><a href="https://github.com/timeseriesAI/tsai/tree/main/tsai/models/TabModel.py#L44" class="source_link" style="float:right">[source]</a></h2><blockquote><p><code>TabHead</code>(<strong><code>emb_szs</code></strong>, <strong><code>n_cont</code></strong>, <strong><code>c_out</code></strong>, <strong><code>layers</code></strong>=<em><code>None</code></em>, <strong><code>fc_dropout</code></strong>=<em><code>None</code></em>, <strong><code>y_range</code></strong>=<em><code>None</code></em>, <strong><code>use_bn</code></strong>=<em><code>True</code></em>, <strong><code>bn_final</code></strong>=<em><code>False</code></em>, <strong><code>lin_first</code></strong>=<em><code>False</code></em>, <strong><code>act</code></strong>=<em><code>ReLU(inplace=True)</code></em>, <strong><code>skip</code></strong>=<em><code>False</code></em>) :: <code>Module</code></p>
</blockquote>
<p>Basic head for tabular data.</p>

</div>

</div>

</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">path</span> <span class="o">=</span> <span class="n">untar_data</span><span class="p">(</span><span class="n">URLs</span><span class="o">.</span><span class="n">ADULT_SAMPLE</span><span class="p">)</span>
<span class="n">df</span> <span class="o">=</span> <span class="n">pd</span><span class="o">.</span><span class="n">read_csv</span><span class="p">(</span><span class="n">path</span><span class="o">/</span><span class="s1">&#39;adult.csv&#39;</span><span class="p">)</span>
<span class="c1"># df[&#39;salary&#39;] = np.random.rand(len(df)) # uncomment to simulate a cont dependent variable</span>
<span class="n">procs</span> <span class="o">=</span> <span class="p">[</span><span class="n">Categorify</span><span class="p">,</span> <span class="n">FillMissing</span><span class="p">,</span> <span class="n">Normalize</span><span class="p">]</span>
<span class="n">cat_names</span> <span class="o">=</span> <span class="p">[</span><span class="s1">&#39;workclass&#39;</span><span class="p">,</span> <span class="s1">&#39;education&#39;</span><span class="p">,</span> <span class="s1">&#39;marital-status&#39;</span><span class="p">,</span> <span class="s1">&#39;occupation&#39;</span><span class="p">,</span> <span class="s1">&#39;relationship&#39;</span><span class="p">,</span> <span class="s1">&#39;race&#39;</span><span class="p">]</span>
<span class="n">cont_names</span> <span class="o">=</span> <span class="p">[</span><span class="s1">&#39;age&#39;</span><span class="p">,</span> <span class="s1">&#39;fnlwgt&#39;</span><span class="p">,</span> <span class="s1">&#39;education-num&#39;</span><span class="p">]</span>
<span class="n">y_names</span> <span class="o">=</span> <span class="p">[</span><span class="s1">&#39;salary&#39;</span><span class="p">]</span>
<span class="n">y_block</span> <span class="o">=</span> <span class="n">RegressionBlock</span><span class="p">()</span> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">df</span><span class="p">[</span><span class="s1">&#39;salary&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">values</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="nb">float</span><span class="p">)</span> <span class="k">else</span> <span class="n">CategoryBlock</span><span class="p">()</span>
<span class="n">splits</span> <span class="o">=</span> <span class="n">RandomSplitter</span><span class="p">()(</span><span class="n">range_of</span><span class="p">(</span><span class="n">df</span><span class="p">))</span>
<span class="n">pd</span><span class="o">.</span><span class="n">options</span><span class="o">.</span><span class="n">mode</span><span class="o">.</span><span class="n">chained_assignment</span><span class="o">=</span><span class="kc">None</span>
<span class="n">to</span> <span class="o">=</span> <span class="n">TabularPandas</span><span class="p">(</span><span class="n">df</span><span class="p">,</span> <span class="n">procs</span><span class="o">=</span><span class="n">procs</span><span class="p">,</span> <span class="n">cat_names</span><span class="o">=</span><span class="n">cat_names</span><span class="p">,</span> <span class="n">cont_names</span><span class="o">=</span><span class="n">cont_names</span><span class="p">,</span> <span class="n">y_names</span><span class="o">=</span><span class="n">y_names</span><span class="p">,</span> <span class="n">y_block</span><span class="o">=</span><span class="n">y_block</span><span class="p">,</span> <span class="n">splits</span><span class="o">=</span><span class="n">splits</span><span class="p">,</span> <span class="n">inplace</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span> 
                   <span class="n">reduce_memory</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
<span class="n">to</span><span class="o">.</span><span class="n">show</span><span class="p">(</span><span class="mi">5</span><span class="p">)</span>
<span class="n">tab_dls</span> <span class="o">=</span> <span class="n">to</span><span class="o">.</span><span class="n">dataloaders</span><span class="p">(</span><span class="n">bs</span><span class="o">=</span><span class="mi">16</span><span class="p">,</span> <span class="n">val_bs</span><span class="o">=</span><span class="mi">32</span><span class="p">)</span>
<span class="n">b</span> <span class="o">=</span> <span class="n">first</span><span class="p">(</span><span class="n">tab_dls</span><span class="o">.</span><span class="n">train</span><span class="p">)</span>
<span class="n">test_eq</span><span class="p">((</span><span class="n">b</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">shape</span><span class="p">,</span> <span class="n">b</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span><span class="o">.</span><span class="n">shape</span><span class="p">,</span> <span class="n">b</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span><span class="o">.</span><span class="n">shape</span><span class="p">),</span> <span class="p">(</span><span class="n">torch</span><span class="o">.</span><span class="n">Size</span><span class="p">([</span><span class="mi">16</span><span class="p">,</span> <span class="mi">7</span><span class="p">]),</span> <span class="n">torch</span><span class="o">.</span><span class="n">Size</span><span class="p">([</span><span class="mi">16</span><span class="p">,</span> <span class="mi">3</span><span class="p">]),</span> <span class="n">torch</span><span class="o">.</span><span class="n">Size</span><span class="p">([</span><span class="mi">16</span><span class="p">,</span> <span class="mi">1</span><span class="p">])))</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">

<div class="output_area">


<div class="output_html rendered_html output_subarea ">
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>workclass</th>
      <th>education</th>
      <th>marital-status</th>
      <th>occupation</th>
      <th>relationship</th>
      <th>race</th>
      <th>education-num_na</th>
      <th>age</th>
      <th>fnlwgt</th>
      <th>education-num</th>
      <th>salary</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>19337</th>
      <td>Private</td>
      <td>HS-grad</td>
      <td>Married-civ-spouse</td>
      <td>Adm-clerical</td>
      <td>Wife</td>
      <td>White</td>
      <td>False</td>
      <td>25.0</td>
      <td>158662.0</td>
      <td>9.0</td>
      <td>&gt;=50k</td>
    </tr>
    <tr>
      <th>15800</th>
      <td>Private</td>
      <td>Some-college</td>
      <td>Married-civ-spouse</td>
      <td>Farming-fishing</td>
      <td>Husband</td>
      <td>White</td>
      <td>False</td>
      <td>54.0</td>
      <td>185407.0</td>
      <td>10.0</td>
      <td>&lt;50k</td>
    </tr>
    <tr>
      <th>715</th>
      <td>Private</td>
      <td>Some-college</td>
      <td>Married-civ-spouse</td>
      <td>#na#</td>
      <td>Wife</td>
      <td>Asian-Pac-Islander</td>
      <td>True</td>
      <td>26.0</td>
      <td>77698.0</td>
      <td>10.0</td>
      <td>&lt;50k</td>
    </tr>
    <tr>
      <th>25329</th>
      <td>Private</td>
      <td>Some-college</td>
      <td>Never-married</td>
      <td>Other-service</td>
      <td>Other-relative</td>
      <td>White</td>
      <td>False</td>
      <td>23.0</td>
      <td>176486.0</td>
      <td>10.0</td>
      <td>&lt;50k</td>
    </tr>
    <tr>
      <th>7389</th>
      <td>Private</td>
      <td>Assoc-voc</td>
      <td>Never-married</td>
      <td>Exec-managerial</td>
      <td>Not-in-family</td>
      <td>Black</td>
      <td>False</td>
      <td>26.0</td>
      <td>491862.0</td>
      <td>11.0</td>
      <td>&lt;50k</td>
    </tr>
  </tbody>
</table>
</div>

</div>

</div>
</div>

</div>
    {% endraw %}

    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">
<div class="input">

<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">tab_model</span> <span class="o">=</span> <span class="n">build_tabular_model</span><span class="p">(</span><span class="n">TabModel</span><span class="p">,</span> <span class="n">dls</span><span class="o">=</span><span class="n">tab_dls</span><span class="p">)</span>
<span class="n">b</span> <span class="o">=</span> <span class="n">first</span><span class="p">(</span><span class="n">tab_dls</span><span class="o">.</span><span class="n">train</span><span class="p">)</span>
<span class="n">test_eq</span><span class="p">(</span><span class="n">tab_model</span><span class="o">.</span><span class="n">to</span><span class="p">(</span><span class="n">b</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">device</span><span class="p">)(</span><span class="o">*</span><span class="n">b</span><span class="p">[:</span><span class="o">-</span><span class="mi">1</span><span class="p">])</span><span class="o">.</span><span class="n">shape</span><span class="p">,</span> <span class="p">(</span><span class="n">tab_dls</span><span class="o">.</span><span class="n">bs</span><span class="p">,</span> <span class="n">tab_dls</span><span class="o">.</span><span class="n">c</span><span class="p">))</span>
<span class="n">learn</span> <span class="o">=</span> <span class="n">Learner</span><span class="p">(</span><span class="n">tab_dls</span><span class="p">,</span> <span class="n">tab_model</span><span class="p">,</span> <span class="n">splitter</span><span class="o">=</span><span class="n">ts_splitter</span><span class="p">)</span>
<span class="n">p1</span> <span class="o">=</span> <span class="n">count_parameters</span><span class="p">(</span><span class="n">learn</span><span class="o">.</span><span class="n">model</span><span class="p">)</span>
<span class="n">learn</span><span class="o">.</span><span class="n">freeze</span><span class="p">()</span>
<span class="n">p2</span> <span class="o">=</span> <span class="n">count_parameters</span><span class="p">(</span><span class="n">learn</span><span class="o">.</span><span class="n">model</span><span class="p">)</span>
<span class="n">learn</span><span class="o">.</span><span class="n">unfreeze</span><span class="p">()</span>
<span class="n">p3</span> <span class="o">=</span> <span class="n">count_parameters</span><span class="p">(</span><span class="n">learn</span><span class="o">.</span><span class="n">model</span><span class="p">)</span>
<span class="k">assert</span> <span class="n">p1</span> <span class="o">==</span> <span class="n">p3</span>
<span class="k">assert</span> <span class="n">p1</span> <span class="o">&gt;</span> <span class="n">p2</span> <span class="o">&gt;</span> <span class="mi">0</span>
</pre></div>

    </div>
</div>
</div>

</div>
    {% endraw %}

</div>
 

